A crucial requirement for a neural network model to be of practical use is its stability. Neural network models whose global stability can be proved are usually described by sets of first order ordinary differential equations of the type ##EQU1## where F.sub.P (t) and D.sub.PQ (t) are the short and the long term memory traces, respectively. Function G(y;z) is generally linear in the second argument; the summation is over the processors Q which are the nearest neighbors of processor P; T.sub.PQ typically are fixed coefficients taking both positive and negative values, and function s'(F.sub.P) is generally of sigmoid type, i.e., it has a threshold, it is monotonic, and it is bounded. Function H is typically some implementation of Hebbian learning rule.
Such equations were employed for a number of methods for pattern recognition, for construction of content addressable memory, and for determining good solutions of complex optimization problems.
Usually the hardware implementations of the neural network theories consist of digital, or software simulations of the neural network differential equations. Such techniques are time-intensive when used on serial digital computers and, therefore, are slow and expensive. For high speed information exchange among the processors it is advantageous to use parallel interconnections and analog processes to carry the information.
Various attempts have been made to produce analog neural network hardware. One method involves dynamical holograms for redirection of optical signal emitted by coherent sources of the processors. The obvious disadvantage of this method is the difficulty in manufacturing the required integrated optoelectronics and in creation of high quality dynamical hologram while maintaining the information about the processors' interconnections stored in it.
Another means to achieve hardware implementation, known in the art, involves manufacture of networks of resistors on a chip. Disadvantages of this method are that the connectivity patterns are constant and once set are impossible to change. In addition, interconnections among the processors take much space on the chip. Finally, because of the general fanout limitations in digital electronics, the fanout for fixed resistor networks can not be expected to be much larger than 100. For comparison the human brain fanout reaches 100,000.
Therefore, there exists a need for simple, cost effective, and energy efficient means for analog, parallel implementation of neural networks with changeable interconnection pattern and large fanout.